Memory device signal processing apparatus image signal processing apparatus and methods of the same

ABSTRACT

A memory device, a signal processing apparatus and an image signal processing apparatus and signal processing methods capable of performing matching processing with a small amount of calculation and accurately detecting motion vectors, provided with an ME memory  13  for converting by using a feature including a pixel value in a certain block range having a focused pixel at its center as an address based on information of a reference frame Fr stored in a second frame memory  12 ; and a matching portion  14  for reading storage information of the ME memory  13  by using a feature of a focused pixel included in information of a current frame Fc supplied from a first frame memory  11  as a feature address, calculating a distance between the focused pixel in the current frame and a feature address (position coordinates) read from the ME memory  13 , and detecting differential coordinates based on position information having the minimum distance from a plurality of candidates as motion vectors (Vx, Vy) of the focused pixel.

TECHNICAL FIELD

[0001] The present invention relates to a memory device, a signalprocessing apparatus, an image signal processing apparatus and signalprocessing methods applicable to processing requiring matching, such asa motion detection device and an object search device, etc. used, forexample, in a motion image compression apparatus, etc.

BACKGOUND ART

[0002] A matching processing of a signal, particularly, a matchingprocessing of an image signal has a problem of generally causing a largeamount of calculation. A motion vector detection by block matching asdescribed below is an example of using the matching processing.

[0003] Namely, in an image signal processing apparatus, motion detectionfor obtaining a motion vector indicating a motion of an image is onemajor technique for effectively performing the motion image compression.There are some proposed methods of obtaining the motion vector, but onemajor method is a method called block a matching algorithm.

[0004]FIG. 1 is a block diagram showing an example of a configuration ofa motion detection device in an image signal processing apparatus of therelated art, wherein the block matching algorithm is applied.

[0005] The motion detection device 1 comprises frame memories 2 and 3and a motion vector detection portion 4.

[0006] In the motion detection device 1, when an image signal is inputfrom an input terminal TIN, an information of one frame is stored in theframe memory 2.

[0007] When an information of the next image is input, the previous(previously input) information in the frame memory 2 is stored in theframe memory 3, and the currently (this time) input information isstored in the frame memory 2.

[0008] Namely, an information of a current frame Fc is stored in theframe memory 2, and an information of a reference frame Fr is stored inthe frame memory 3.

[0009] Next, the information of the current frame Fc and the referenceframe Fr is sent to the vector detection portion 4 and divided to blocksin the motion vector detection portion 4 where a motion vector (Vx, Vy)is detected and output from a terminal TOUT.

[0010]FIG. 2 is a view for explaining an outline of block matchingalgorithm. Below, the outline of algorithm will be explained withreference to FIG. 2.

[0011] In this algorithm, a motion vector of a focused pixel Fc (x, y)in the current frame Fc is obtained by calculating a differentialabsolute value sum of corresponding pixels in pixels in a referenceblock range (L×L) having the focused pixel Fc (x, y) at its center andpixels in the same block range as the above block range (L×L) in asearch area SR in the reference frame Fr.

[0012] The above calculation is repeated while moving the block rangeextracted in the search area SR by one pixel, and a differential vectorof the center position of a block having the smallest differentialabsolute value sum in all blocks and the focused pixel position isregarded as the solution (motion vector).

[0013] Next, a processing procedure of detecting a motion vector of thepixel Fc (x, y) in the current frame Fc will be explained in detail withreference to FIG. 3.

[0014] Step ST1:

[0015] In the step ST1, a search area SR using as a reference the sameposition in the reference frame as the position (x, y) of the focusedpixel is determined after starting processing ST0.

[0016] Step ST2:

[0017] In the step ST2, the maximum value of a calculation formula issubstituted to initialize a variable “min” storing the minimum value ofa calculation result. When assuming that one pixel is 8 bits and thenumber of pixels in a block is 16, 2^(8×16=4096) is assigned to thevariable “min”.

[0018] Step ST3:

[0019] In the step ST3, a counter variable “n” for counting blocks in asearch area SR is initialized to be

[0020] Step ST4:

[0021] In the step ST4, a variable “sum” for being substituted acalculation result is initialized to be “0”.

[0022] Step ST5:

[0023] In the step ST5, assuming that a range of a reference block isL×L, a pixel in a certain block in the current frame is Fc (i, j), and apixel in the k-th block in the search area SR of the reference frame Fris Frk (i, j), calculation of a differential absolute value sum ofcorresponding pixels, that is, the formula 1 below is performed and thecalculation result is substituted for $\begin{matrix}{{\sum\limits_{i = 1}^{L}\sum\limits_{j = 1}^{L}} = {{{{Fc}\left( {i,j} \right)} - {{Frk}\left( {i,j} \right)}}}} & (1)\end{matrix}$

[0024] Step ST6

[0025] In the step ST6, the relationship of large and small isdistinguished between the calculated differential absolute value sum“sum” and the minimum value “min” of the differential absolute valuesum. When the calculated differential absolute value sum “sum” issmaller, the procedure proceeds to a step ST7, while when it is larger(including being equal), the calculation result is not the minimumvalue, so that the step ST7 is skipped and the procedure proceeds to astep ST8.

[0026] Step ST7:

[0027] In the step ST7, the calculation result “sum” is updated by theminimum value “min”, and a counter value “n” of the block as a motionvector number is set.

[0028] Step ST8:

[0029] In the step ST8, if the block counter value “n” is the totalnumber of blocks in the search area SR, that is, the final block comes,it means the finish, so the procedure proceeds to a step ST10, while ifit is not the final block, the procedure proceeds to a step ST9.

[0030] Step ST9:

[0031] In the step ST9, the block counter value “n” is incremented to“n+1”, and the procedure proceeds to the step ST4 to repeat thecalculation.

[0032] Step ST10:

[0033] In the step ST10, a motion vector is obtained from the centerpixel and (x, y) of a block having a number stored in a motion numberand output the same.

[0034] Since the block matching algorithm explained above repeats thecalculation of the formula (1), there is a disadvantage that an amountof the calculation becomes enormous and most of the time of imagecompression processing, such as MPEG, is spent for that.

DISCLOSURE OF THE INVENTION

[0035] An object of the present invention is to provide a memory device,a signal processing apparatus, an image processing apparatus and signalprocessing methods capable of performing matching processing, etc. onlywith a small amount of calculation and accurately detecting a motionvector, etc.

[0036] To attain the above object, a first aspect of the presentinvention is a memory device for storing information, characterized bycomprising an input/output means for receiving or outputting theinformation; and a storage portion for storing the information; whereinthe storage portion is at least controlled by an address correspondingto a feature of a focused data in a predetermined signal; and theinformation is a position information of the focused data in thepredetermined signal.

[0037] Preferably, the memory device is a semiconductor device.

[0038] A second aspect of the present invention is a signal processingapparatus for performing matching processing by using a first signalincluding a plurality of first data and a second signal including aplurality of second data, characterized by comprising a first featureextraction means for extracting a feature as focused data being data ata focused position in the first signal; and a storage means for storingposition information of the second data at a position specified by anaddress corresponding to the feature of each of the second data; whereinthe position information of the second data corresponding to the focuseddata is obtained by reading the position information to the second datastored in the storage means at an address corresponding to the featureof the focused data.

[0039] Preferably, the storage means stores the position information ofthe second data in advance.

[0040] Also, the above signal processing apparatus further comprises asecond feature extraction means for extracting a feature of each ofsecond data in the second signal.

[0041] Preferably, the first feature extraction means further comprisesa second feature extraction means for extracting a feature of each ofsecond data in the second signal.

[0042] Also, the first or second signal is an image signal.

[0043] Also, the feature is extracted by using a predetermined number offirst or second data.

[0044] Also, the feature is a distribution of values of a predeterminednumber of first or second data.

[0045] Also, the first or second data is expressed by a plurality ofbits, and the feature is a distribution of values of the predeterminednumber of first or second data wherein predetermined bits are omittedfrom the plurality of bits.

[0046] Preferably, the predetermined bits are lower side bits of thefirst or second data.

[0047] Also, the feature is a distribution of values of predeterminednumber of re-quantized first or second data. A feature is a distributionof values of predetermined number of first or second data subjected toadaptive quantization. Also, the adaptive quantization is an ADRC.

[0048] Preferably, when a plurality of position information of thesecond data corresponding to a feature of the focused data are read fromthe storage means, the position information having the shortest distanceto a position of the focused data is regarded as position information ofthe second data corresponding to the focused data.

[0049] A third aspect of the present invention is an image signalprocessing device for detecting motion vectors by using a first imagesignal including a plurality of first data and a second image signalincluding a plurality of second data, characterized by comprising afirst feature extraction means for extracting a feature as focused databeing data at a focused position in the first image signal; a storagemeans for storing position information of the second data at a positionspecified by an address corresponding to the feature of each of thesecond data; and a motion vector calculation means for obtainingposition information of the second data corresponding to the focuseddata by reading position information of the second data stored in thestorage means at an address corresponding to the feature of the focuseddata and calculating a motion vector of the focused data by usingposition information of the focused data and the obtained positioninformation of the second data.

[0050] Preferably, when a plurality of position information of thesecond data corresponding to a feature of the focused data are read fromthe storage means, the motion vector calculation means regards theposition information having the shortest distance to a position of thefocused data as position information of the second data corresponding tothe focused data.

[0051] A fourth aspect of the present invention is a signal processingmethod for performing matching processing by using a first signalincluding a plurality of data and a second signal including a pluralityof second data, comprising a first step for extracting a feature asfocused data being data at a focused position in the first signal; asecond step for storing position information of the second data at aposition specified by an address corresponding to the feature of each ofthe second data; and a third step for obtaining position information ofthe second data corresponding to the focused data by reading positioninformation to the second data stored in the second step at an addresscorresponding to the feature of the focused data.

[0052] A fifth aspect of the present invention is an image signalprocessing method for detecting a motion vector by using a first imagesignal including a plurality of first data and a second image signalincluding a plurality of second data, comprising a first step forextracting a feature as a focused data being data of a focused positionin the first images signal; a second step for storing positioninformation of the second data at a position specified by an addresscorresponding to the feature of each of the second data; a third stepfor obtaining position information of the second data corresponding tothe focused data by reading position information to the second datastored in the storage means at an address corresponding to the featureof the focused data; and a fourth step for calculating a motion vectorof the focused data by using position information of the focused dataand the obtained position information of the second data.

BRIEF DESCRIPTION OF DRAWINGS

[0053]FIG. 1 is a block diagram showing an example of a configuration ofa motion detection device of the related art wherein block matchingalgorithm is applied.

[0054]FIG. 2 is a view for explaining an outline of block matchingalgorithm.

[0055]FIG. 3 is a flowchart for explaining a processing procedure ofdetecting a motion vector of a pixel Fc (x,

[0056] y) in a current frame FC.

[0057]FIG. 4 is a block diagram showing a signal processing apparatushaving a basic configuration of a motion detection device as a principalpart of an image signal processing apparatus according to the presentinvention.

[0058]FIG. 5 is a block diagram showing an example of a basicconfiguration of first and second data storing portions in FIG. 4.

[0059]FIG. 6 is a block diagram showing an example of a configuration ofa matching portion in FIG. 4.

[0060]FIG. 7 is a block diagram showing a first embodiment of a motiondetection device according to the present invention.

[0061]FIG. 8 is a flowchart showing an outline of motion vectordetection according to the present embodiment.

[0062]FIG. 9 is a view for explaining the configuration of a motionmemory in a feature address system according to the present embodiment.

[0063]FIG. 10 is a view for explaining a procedure of storing in themotion memory in a feature address system according to the presentembodiment.

[0064]FIG. 11 is a flowchart for explaining an operation of motiondetection in a feature address system according to the presentembodiment.

[0065]FIG. 12 is a block diagram showing a second embodiment of a motiondetection device according to the present invention.

[0066]FIG. 13 is a block diagram showing another example of aconfiguration of a motion detection device according to the secondembodiment.

[0067]FIG. 14A to FIG. 14D are views showing an example of taking aclass tap.

[0068]FIG. 15A and FIG. 15B are views for explaining the fact that usingan ADRC quantization code is superior to using a surrounding pixelvalue.

[0069]FIG. 16 is a view showing a correspondence relationship of decimalnumber and hexadecimal number of a luminance value.

BEST MODE FOR CARRYING OUT THE INVENTION

[0070] Below, preferred embodiments of the present invention will beexplained with reference to the attached drawings.

[0071]FIG. 4 is a block diagram showing a signal processing devicehaving a basic configuration of a motion detection device as a principalpart of an image signal processing apparatus according to the presentinvention.

[0072] The signal processing apparatus is for accurately estimatingposition information (a motion vector) with a small amount ofcalculation by performing processing of obtaining (determining) positioninformation of second data corresponding to focused data by reading theposition information of second data included in a second signal storedin a motion detection memory (hereinafter, referred to as an ME memory)at an address corresponding to a feature of the focused data being dataat a focused position in a first signal including a plurality of firstdata.

[0073] Note that the ME memory (a feature address, i) indicates datastored at a position specified by an address (a feature address, i) inthe ME memory.

[0074] Below, the specific configuration and function of the signalprocessing apparatus will be explained in detail with reference to thedrawings.

[0075] The signal processing apparatus 10 comprises a first data storingportion 11, a second data storing portion 12, an ME memory 13 and amatching portion 14.

[0076] Note that the first data storing portion 11 configures a firstfeature extraction means according to the present invention, the seconddata storing portion 12 configures a second feature extraction meansaccording to the present invention, the ME memory 13 configures astorage means according to the present invention, and the matchingportion 14 configures a motion vector calculation means (matching means)according to the present invention.

[0077] The first data storing portion 11 stores a first signal includinga plurality of first data input from an input terminal TIN.

[0078] The first data storing portion 11 receives as an input the nextfirst signal and stores this next first signal and outputs thepreviously stored first signal as a second signal including a pluralityof second data to the second data storing portion 12 and the matchingportion 14.

[0079] Also, the first data storing portion 11 extracts a feature of afocused data being data at a focused position from the stored firstsignal and supplies feature information including position informationof the focused data to the matching portion 14.

[0080] The second data storing portion 12 stores the previous (forexample, one time before) second signal stored in the first data storingportion 11, extracts a feature of each of the second data from thestored second signal and stores in the ME memory 13.

[0081]FIG. 5 is a block diagram showing an example of a configuration ofthe first and second data storing portions. As shown in FIG. 5, thefirst and second data storing portions respectively comprises a memory111 and a feature extraction portion 112 for extracting a feature fromthe first or second signal stored in the memory 111.

[0082] The ME memory 13 receives features of second data of the secondsignal in the second data storing portion 12 and stores positioninformation of the second data at positions specified by addressescorresponding to the features of the second data.

[0083] The matching portion 14 obtains (determines) position informationof second data corresponding to the focused data by reading the positioninformation of the second data included in the second signal stored inthe ME memory 13 at an address corresponding to a feature of the focuseddata being data at a focused position input from the first data storingportion 11.

[0084] When a plurality of position information of the second datacorresponding to a feature of the focused data is read from the MEmemory 13, the matching portion 14 ragards the above positioninformation closest to a position of the focused data as positioninformation of second data corresponding to the focused data.

[0085]FIG. 6 is a block diagram showing an example of a configuration ofa matching portion 14.

[0086] The matching portion 14 comprises, as shown in FIG. 6, an inputportion 141, a read portion 142 for reading the position information ofthe second data included in the second signal stored in the ME memory 13at an address corresponding to a feature of the focused data being dataof a focused position based on feature information from the inputportion 141, and a determination portion 143 for determining positioninformation of the second data corresponding to the focused data basedon the position information of the first data from the input portion 141and the information read from the ME memory 13. When used in motiondetection, the determination portion 413 determines as a motion vectorand output the same.

[0087] Note that the first and second signals are, for example, imagesignals.

[0088] Also, for example, feature extraction in the first and seconddata storing portions 11 and 12 is performed by using predeterminednumber of first and second data. Also, the feature is a distribution ofvalues of the predetermined number of first or second data. Alternately,the first and second data is expressed by a plurality of bits, and thefeature is a distribution of values of the above predetermined number offirst or second data excluding predetermined bits from the aboveplurality of bits. Here, the predetermined bits are lower side bits ofthe first or second data. Also, the feature is a distribution of valuesof predetermined number of re-quantized first or second data.Alternately, the feature is a distribution of values of predeterminednumber of first or second data subjected to adaptive quantization, andthe adaptive quantization is ADRC (Adaptive Dynamic Range Coding).

[0089] Below, embodiments of a motion detection device as a principalpart of an image signal processing apparatus based on a signalprocessing apparatus having the above configuration will be explained.

[0090] First Embodiment

[0091]FIG. 7 is a block diagram showing a first embodiment of a motiondetection device as a principal part of an image processing apparatusaccording to the present invention.

[0092] The motion detection device is provided with an ME memory forstoring position information by using a feature as an address andperforms matching processing by using a surrounding pixel value as afeature, so that motion vectors can be accurately estimated with alittle amount of calculation.

[0093] Below, the specific configuration and function of the motiondetection device will be explained in detail with reference to thedrawings.

[0094] The motion detection device 20 comprises a second frame memory 21as a first data storing portion and a second frame memory 22 as a seconddata storing portion, an ME memory 23 and a matching portion 24 as amatching portion.

[0095] Note that the first frame memory 21 configures the first featureextraction means according to the present invention, the second framememory 22 configures the second feature extraction means according tothe present invention, the ME memory 23 configures the storing meansaccording to the present invention, and the matching portion 24configures the motion vector calculation means (matching means)according to the present invention.

[0096] The first frame memory 21 stores an information of one frame ofan image signal input from an input terminal TIN.

[0097] When the next frame information is input, the first frame memory21 stores previously stored frame information, that is, the informationof the current frame Fc and outputs the information of the current frameFc to the second frame memory 22 and the matching portion 24.

[0098] Also, the first frame memory 21 supplies a feature of the focusedpixel, that is, address information together with information of thecurrent frame Fc to the matching portion 24.

[0099] The second frame memory 22 stores previous (for example, one timebefore) frame information stored in the first frame memory 21 asinformation of a reference frame Fr.

[0100] The ME memory 23 stores information including positioncoordinates of the focused pixel corresponding to a feature by using thefeature as an address as a pixel value of a certain block range havingthe focused pixel at its center based on the information of thereference frame Fr stored in the second frame memory 22.

[0101] The matching portion 24 reads storage information in the MEmemory 23 by using the feature of the focused pixel included in theinformation of the current frame Fc supplied from the first frame memory21 as a feature address. Differential coordinates based on the focusedpixel in the current frame and the stored information read from the MEmemory 23 are output as a motion vector (Vx, Vy) of the focused pixelfrom a terminal TOUT. Note that when there are a plurality of storedinformation (position coordinates) in a feature address corresponding tothe feature of the focused pixel, a motion vector by which a positionand distance of the focused pixel become the minimum among the pluralityof candidates is output.

[0102]FIG. 8 is a flowchart showing an outline of motion vectordetection according to the present embodiment.

[0103] In this example, first, data of the reference frame Fr is storedin the ME memory 23 (step ST21). Next, the matching portion 24 readsdata from the ME memory 13 at a feature address in accordance with afeature of a pixel Ln in the current frame Fc (Step ST22).

[0104] The matching portion 24 determines the position information ofdata corresponding to the pixel Ln in the read data (step ST 23). Then,motion vectors are calculated until motion vectors of all pixels areobtained (steps ST23 and ST24).

[0105] Below, the configuration and function of the ME memory 23 whereina feature address system as a characteristic of the present embodimentwill be explained further in detail with reference to FIG. 9 and FIG.10.

[0106]FIG. 9 is a view showing an example of a configuration of the MEmemory wherein the feature address system is applied. Also, FIG. 10 is aflowchart for explaining a procedure of storing information of thereference frame in the ME memory.

[0107] The conventional memory is for storing pixel value by using theposition information of a pixel as an address, while the ME memory 23stores for every feature position information of a pixel having thefeature in flag addresses FRGA1, 2, . . . , that is, B, C . . . in FIG.9 successively.

[0108] In the present embodiment, one cell ME-B1 has a storage capacityof an amount of position information. Also, the number of positioninformation stored for the feature is stored in a flag address FRGA0.

[0109] The feature is a pixel value in a certain block having a focusedpixel at its center. For example, when assuming that a block range is3×3, the vertical direction is “i”, the horizontal direction is “j”, anda pixel value of a position (i, j) is L(i, j), a feature in this casebecomes as in the next formula (2).

{L(i−1, j−1), L(i−1, j), L(i−1, j+1), L(i, j−1), L(i, j), L(i, j+1),L(i+1, j−1), L(i+1, j), L(i+1, j+1)}  (2)

[0110] Next, a procedure of storing information of the reference frameto the ME memory will be explained with reference to the flowchart inFIG. 10.

[0111] When information of the reference frame Fr is stored in the framememory 22, the processing starts.

[0112] Step ST101:

[0113] In the step ST101, all data in the ME memory is initialized to be“0” by writing “0” or turning on a reset signal.

[0114] Step ST102:

[0115] In the step ST102, a counter variable “n” for counting pixels inone frame memory is initialized to be “0”.

[0116] Step ST103:

[0117] In the step ST103, a pixel value of a certain block range havingthe focused pixel Ln at its center is regarded as a feature (a featureaddress) in the frame memory 21 in FIG. 4.

[0118] Step ST104:

[0119] In the step ST104, a content in the ME memory (a feature, 0) isincremented by one.

[0120] Step ST105:

[0121] A feature address is set to be the feature in the step ST103 andthe ME memory (a feature, 0) as a content of the ME memory 23 when aflag address is “0” is set to a read flag address. In the step ST105,the position information of the focused pixel Ln is written to a contentof the ME memory (a feature, flag address) as a content of the ME memory13.

[0122] Step ST106:

[0123] In the step ST106, the counter variable “n” is incremented.

[0124] Step ST107:

[0125] In the step ST107, whether the focused pixel Ln is the finalpixel in the frame or not is judged. When it is not the final pixel, theprocedure proceeds to the step ST103 and the same processing is repeatedon the next pixel.

[0126] While, when it is the final pixel, the procedure proceeds to astep ST109 to finish the processing.

[0127] Next, a processing procedure of motion vector detection accordingto the present embodiment will be explained with reference to FIG. 7 andFIG. 11.

[0128] Note that FIG. 11 is a flowchart for explaining the processingprocedure of motion vector detection according to the presentembodiment.

[0129] Step ST201:

[0130] In the step ST201, after storing information of the current frameFc and the reference frame Fr respectively in the frame memories 21 and22, information of the reference frame is stored in the ME memory 23while converting it to a feature address. The details are explainedabove (the steps ST100 to ST109).

[0131] Step ST202:

[0132] In the step ST202, the counter variable “n” for counting pixelsin one frame is initialized to be “0”.

[0133] Step ST203:

[0134] In the step ST203, a feature of the focused pixel Ln in the firstcurrent frame 21 is a pixel value of a certain block range having thefocused pixel at its center, so it is sent as a feature to the matchingportion 24.

[0135] Step ST204:

[0136] In the step ST204, the matching portion 24 reads a value of acontent ME memory (a feature address, 0) from the ME memory 23 by usingthe received feature as a feature address and substitutes for a variable“kn” meaning the number of options.

[0137] Also, a variable “k” meaning a counter of the number of optionsis initialized to be “1”, a variable “min” meaning the minimum value ofa distance is initialized to be ∞ or a sufficiently large value, and avariable “L” meaning a distance is initialized to be “0”.

[0138] Step ST205:

[0139] In the step ST205, a distance between the focused pixel Ln in thecurrent frame and an ME memory content (a feature address, k)=positioncoordinates read from the ME memory 23 is calculated and substituted forthe variable “L”.

[0140] Step ST206:

[0141] In the step ST206, the distinction of large and small between thedistance L obtained in the step ST205 and the minimum value “min” of thedistance are performed.

[0142] As a result, when min>L, the procedure proceeds to a step ST207for updating the minimum value L of the distance, while when min<L, theupdate step is skipped and the procedure proceeds to a step ST208.

[0143] Step ST207:

[0144] In the step ST207, the minimum value “min” of the distance isupdated to be L. At this time, a flag address value, that is, “k” isstored to a variable “ans”.

[0145] Step ST208:

[0146] In the step ST208, whether the candidate counter indicates thenumber of candidates or not is judged, and when it is the number ofcandidates, the procedure proceeds to a step ST210, while when there arestill other candidates, the procedure proceeds to the step ST209.

[0147] Step ST209:

[0148] In the step ST209, the candidate counter “k” is incremented,then, the procedure proceeds to the step ST 205.

[0149] Step ST210:

[0150] In the step ST210, the position information having the minimumdistance to the pixel Ln in the current frame, that is, a value of acontent ME memory (a feature address, ans) of the ME memory 23 is readand the differential coordinates are regarded as a motion vector.

[0151] Step ST211:

[0152] In the step ST211, a motion vector of the focused pixel isoutput.

[0153] Step ST212:

[0154] In the step ST 212, a counter variable “n” of pixels isincremented.

[0155] Step ST213:

[0156] In the step ST213, whether the focused pixel is the final pixelin the current frame or not is judged. As a result of the judgment, whenit is the final pixel the procedure proceeds to a step ST214, while not,the procedure proceeds to the step ST203 to obtain a motion vector ofthe next pixel.

[0157] As to a pixel value, for example, when one pixel=8 bits, matchingprocessing can be performed with full bit (8 bits) information on animage such as computer graphics (CG), while, since frames are uneven ina natural image, the matching processing is preferably performed byomitting predetermined bits from a plurality of bits. Specifically,several low bits may be masked for use or the number of bits may bedecreased for re-quantization.

[0158] Namely, it is preferable to decrease the number of bits innonlinear/linear quantization (decrease the number of quantizationbits).

[0159] As explained above, according to the present first embodiment, afirst frame memory 21 for storing information of a current frame Fc andoutputting address information as a feature of a focused pixel togetherwith the information of the current frame Fc; a second frame memory 12for storing previous (one time before) frame information stored in thefirst frame memory 21 as information of a reference frame Fr; an MEmemory 23 for converting a feature including a pixel value of a certainblock range having the focused pixel at its center based on theinformation of the reference frame Fr stored in the second frame memory22 and storing information including position information after theconversion; and a matching portion 24 for reading storage information inthe ME memory 23 by using a feature of the focused pixel included in theinformation of the current frame Fc supplied from the first frame memory21 as a feature address, calculating a distance between the focusedpixel in the current frame and the feature address (positioncoordinates) read from the ME memory 23, and detecting the differentialcoordinates based on the position information having the minimumdistance in a plurality of candidates as a motion vector (Vx, Vy) of thefocused pixel, as a result, effects below can be obtained.

[0160] Namely, in the first embodiment, spatial pattern information in ablock area is used as a feature and calculation and comparison of adistance are performed only for exactly the number of candidates, sothat there is an advantage that highly accurate motion vector detectionbecomes possible with a smaller amount of calculation than that in theconventional method.

[0161] Note that when there are a large number of candidates, storageinformation in the ME memory 23 does not have to be the wholeinformation of one frame and may be divided to some areas.

[0162] Second Embodiment

[0163]FIG. 12 is a block diagram showing a second embodiment of a motiondetection device as an image processing apparatus according to thepresent invention.

[0164] A different point in the present second embodiment from the firstembodiment explained above is that matching with a preferable featurebecomes possible by providing class generation portions 25-1 and 25-2 asfeature generation means for obtaining a feature. Note that, as shown inFIG. 13, the configuration of providing one class generation portion 25as in FIG. 13 is also possible. As a function of the circuit, it alsohas functions of the class generation portion 25-1 and 25-2.

[0165] The class generation portion 25-1 receives information of acurrent frame Fc of the first frame memory 21, generates a quantizationcode based on the ADRC as a feature of the current frame and outputs toa matching portion 24A.

[0166] Also, the class generation portion 25-2 receives information of areference frame Fr of the second frame memory 22, generates aquantization code based on the ADRC as a feature of the current frameand outputs to the ME memory 23.

[0167] The matching portion 24A reads storage information of the MEmemory 23 by using a feature of the current frame as a feature addressand performs matching of a focused pixel in a current frame andquantization codes of the ADRC included in the feature address read fromthe ME memory 23 to detect a motion vector of the focused pixel.

[0168] As explained above, the ADRC is used for generating a feature inthe class generation portions 25-1 and 25-2 according to the secondembodiment. The ADRC (Adaptive Dynamic Range Coding) is an adaptivequantization method developed for high performance coding of a VTR(Video Tape Recorder) and is capable of effectively expressing a partialpattern of a signal level with a short word length, so that the ADRC isused for code generation of spatial class division in the secondembodiment.

[0169] The ADRC is an algorithm for performing re-quantization by evenlydividing between the maximum value MAX and the minimum value MIN by aspecified bit length by the formula 3 below, wherein a dynamic range ofa spatial class tap is DR, a bit assignment is n, a data level of apixel of a spatial class tap is L and a re-quantization code is Q.

DR=MAX−MIN+1

Q={(L−MIN+0.5)×2^(n) ÷DR}  (3)

[0170] Note that { } indicates rounding processing. A matchingprocessing flow will be omitted because it is equivalent to that usingthe feature as a quantization code of the ADRC in the above explanationof FIG. 11.

[0171] As an example of taking a spatial class tap, when a block size is3×3, all pixels may be used as shown in FIG. 14A or it may be configuredto be the shape of a cross, and it may be determined within a limit ofan information amount given to the class code.

[0172] Similarly, as an example of the case where the block size is 5×5,embodiments shown in FIG. 14C and FIG. 14D can be applied.

[0173] The example in FIG. 14C is the case of configuring to be theshape of a cross, and the example in FIG. 14D is the case of configuringto be the shape of a cross and, furthermore, using pixels at corners.

[0174] Next, the fact that the case of using a surrounding pixel valueis superior to the case of using the ADRC quantization code will beexplained with reference to FIG. 15A and FIG. 15B.

[0175]FIG. 15A and FIG. 15B show a pixel value when certain one line ofan image is moved from a reference frame to a current frame forfacilitating understanding. Also, FIG. 16 shows a correspondencerelationship of decimal number expression and hexadecimal numberexpression of luminance values.

[0176] Normally, in the case of a natural image, even when the samepattern is moved, there is a low possibility to be the same pixel valueand a pixel level deviates as shown in FIG. 15A and FIG. 15B.

[0177] In this case, the point is whether or not it is possible tocorrectly detect as the same pattern. A coding result in the case ofmasking lower bits to suppress an effect of noise components when usinga surrounding pixel value as a feature is listed.

[0178] As described above, there are some cases of erroneous detectioneven being the same pattern.

[0179] On the other hand, quantization coding by the ADRC is resistibleto small level fluctuations due to capability of effectively expressinga partial pattern of a signal level with a short word length, and thesame coding result can be obtained.

[0180] Specifically, an ADRC code of a certain line of the referenceframe is “01101”, and an ADRC code of a certain line of the currentframe is also “01101”, so that the two match.

[0181] As explained above, according to the second embodiment, by usingthe ADRC quantization code as a feature, there is an advantage thatmotion vector detection at higher accuracy than conventional casebecomes possible.

[0182] In the first and second embodiments, a memory devicecorresponding to motion vector detection was explained. Therefore, amemory device for storing spatial coordinates by using each featherquantity as an address was taken as an example.

[0183] However, the present invention can be applied to systemsperforming matching other than motion vector detection.

[0184] This idea can be expressed “a memory device for storinginformation on first data, characterized by comprising an input meansfor receiving as an input a first feature indicating a first feature(address) of focused data in second data being different from the abovefirst data; and

[0185] a memory means for storing a second feature indicating a secondfeature (coordinates) of the above certain data in the above first dataat a position corresponding to a third feature indicating the abovefirst feature (address) of certain data in the above first data;

[0186] wherein the above second feature as the above second feature ofthe above first data is output from a position corresponding to theabove first feature of the above memory means”.

[0187] Namely, a second feature indicating a feature B (second feature)of data may be stored by using each first feature indicating a feature A(first feature) of data as an address. The feature A and feature B canbe suitably set/changed in accordance with an object of a system/deviceperforming matching. For example, in the present embodiment, anexplanation was made assuming that the feature A: a pixel valuepattern/the ADRC code, and the feature B: coordinates, but otherfeatures may be also used.

INDUSTRIAL APPLICABILITY

[0188] According to a memory device, a signal processing apparatus, animage signal processing apparatus and a signal processing methods of thepresent invention, highly accurate motion vector detection becomespossible, so that they are applicable to processing requiring matching,such as a motion detection device and an object search device used in amoving image compression apparatus, etc.

1. A memory device for storing information, characterized by comprisingan input/output means for receiving or outputting said information; anda storage portion for storing said information; wherein said storageportion is at least controlled by an address corresponding to a featureof a focused data in a predetermined signal; and said information is aposition information of said focused data in said predetermined signal.2. A memory device as set forth in claim 1, characterized in that saidmemory device is a semiconductor device.
 3. A signal processingapparatus for performing matching processing by using a first signalincluding a plurality of first data and a second signal including aplurality of second data, characterized by comprising: a first featureextraction means for extracting a feature as focused data being data ata focused position in said first signal; and a storage means for storingposition information of said second data at a position specified by anaddress corresponding to said feature of each of said second data;wherein: the position information of said second data corresponding tosaid focused data is obtained by reading the position information tosaid second data stored in said storage means at an addresscorresponding to said feature of said focused data.
 4. A signalprocessing apparatus as set forth in claim 3, characterized in that saidstorage means stores the position information of said second data inadvance.
 5. A signal processing apparatus as set forth in claim 3,characterized in that said signal processing apparatus further comprisesa second feature extraction means for extracting a feature of each ofsecond data in said second signal.
 6. A signal processing apparatus asset forth in claim 3, characterized in that said first featureextraction means further comprises a second feature extraction means forextracting a feature of each of second data in said second signal.
 7. Asignal processing apparatus as set forth in claim 3, characterized inthat said first or second signal is an image signal.
 8. A signalprocessing apparatus as set forth in claim 3, characterized in that saidfeature is extracted by using a predetermined number of first or seconddata.
 9. A signal processing apparatus as set forth in claim 8,characterized in that said feature is a distribution of values of apredetermined number of first or second data.
 10. A signal processingapparatus as set forth in claim 9, characterized in that: said first orsecond data is expressed by a plurality of bits; and said feature is adistribution of values of said predetermined number of first or seconddata wherein predetermined bits are omitted from said plurality of bits.11. A signal processing apparatus as set forth in claim 10,characterized in that said predetermined bits are lower side bits ofsaid first or second data.
 12. A signal processing apparatus as setforth in claim 8, characterized in that said feature is a distributionof values of predetermined number of re-quantized first or second data.13. A signal processing apparatus as set forth in claim 8, characterizedin that said feature is a distribution of values of predetermined numberof first or second data subjected to adaptive quantization.
 14. A signalprocessing apparatus as set forth in claim 13, characterized in thatsaid adaptive quantization is an ADRC.
 15. A signal processing apparatusas set forth in claim 3, characterized in that, when a plurality ofposition information of said second data corresponding to a feature ofsaid focused data are read from said storage means, said positioninformation having the shortest distance to a position of said focuseddata is regarded as position information of said second datacorresponding to said focused data.
 16. An image signal processingapparatus for detecting motion vectors by using a first image signalincluding a plurality of first data and a second image signal includinga plurality of second data, characterized by comprising: a first featureextraction means for extracting a feature as focused data being data ata focused position in said first image signal; a storage means forstoring position information of said second data at a position specifiedby an address corresponding to said feature of each of said second data;and a motion vector calculation means for obtaining the positioninformation of said second data corresponding to said focused data byreading the position information of said second data stored in saidstorage means at an address corresponding to said feature of saidfocused data and calculating a motion vector of said focused data byusing position information of said focused data and the obtainedposition information of said second data.
 17. An image signal processingapparatus as set forth in claim 16, characterized in that said imagesignal processing apparatus further comprises a second featureextraction means for extracting a feature of each of second data in saidsecond image signal.
 18. An image signal processing apparatus as setforth in claim 16, characterized in that said first feature extractionmeans further comprises a second feature extraction means for extractinga feature of each of second data in said second image signal.
 19. Animage signal processing apparatus as set forth in claim 16,characterized in that when a plurality of position information of saidsecond data corresponding to a feature of said focused data are readfrom said storage means, said motion vector calculation means regardssaid position information having the shortest distance to a position ofsaid focused data as position information of said second datacorresponding to said focused data.
 20. A signal processing method forperforming matching processing by using a first signal including aplurality of data and a second signal including a plurality of seconddata, characterized by comprising: a first step for extracting a featureas focused data being data at a focused position in said first signal; asecond step for storing position information of said second data at aposition specified by an address corresponding to said feature of eachof said second data; and a third step for obtaining the positioninformation of said second data corresponding to said focused data byreading the position information to said second data stored in saidsecond step at an address corresponding to said feature of said focuseddata.
 21. A signal processing method as set forth in claim 20, furtherincluding a fourth step for extracting a feature of each of second datain said second signal.
 22. A signal processing method as set forth inclaim 20, characterized in that said first step further includes afourth step for extracting a feature of each of second data in saidsecond signal.
 23. A signal processing method as set forth in claim 20,characterized in that said first or second signal is an image signal.24. A signal processing method as set forth in claim 20, characterizedin that said feature is extracted by using a predetermined number offirst or second data.
 25. A signal processing method as set forth inclaim 24, characterized in that said feature is a distribution of valuesof a predetermined number of first or second data
 26. A signalprocessing method as set forth in claim 25, characterized in that: saidfirst or second data is expressed by a plurality of bits; and saidfeature is a distribution of values of said predetermined number offirst or second data wherein predetermined bits are omitted from saidplurality of bits.
 27. A signal processing method as set forth in claim26, characterized in that said predetermined bits are lower side bits ofsaid first or second data.
 28. A signal processing method as set forthin claim 24, characterized in that said feature is a distribution ofvalues of predetermined number of re-quantized first or second data. 29.A signal processing method as set forth in claim 24, characterized inthat said feature is a distribution of values of a predetermined numberof first or second data subjected to adaptive quantization.
 30. A signalprocessing method as set forth in claim 29, characterized in that saidadaptive quantization is an ADRC.
 31. A signal processing device as setforth in claim 20, characterized in that, when a plurality of positioninformation of said second data corresponding to a feature of saidfocused data are read from said storage means, said position informationhaving the shortest distance to a position of said focused data isregarded as position information of said second data corresponding tosaid focused data.
 32. An image signal processing method for detecting amotion vector by using a first image signal including a plurality offirst data and a second image signal including a plurality of seconddata, characterized by comprising: a first step for extracting a featureas a focused data being data of a focused position in said first imagessignal; a second step for storing position information of said seconddata at a position specified by an address corresponding to said featureof each of said second data; a third step for obtaining positioninformation of said second data corresponding to said focused data byreading position information to said second data stored in said storagemeans at an address corresponding to said feature of said focused data;and a fourth step for calculating a motion vector of said focused databy using position information of said focused data and the obtainedposition information of said second data.
 33. An image signal processingmethod as set forth in claim 32, characterized in that said image signalprocessing method further includes a fifth step for extracting a featureof each of second data in said second image signal.
 34. An image signalprocessing method as set forth in claim 32, characterized in that saidfirst step further includes a fifth step for extracting a feature ofeach of second data in said second image signal.
 35. An image signalprocessing method as set forth in claim 32, characterized in that, whena plurality of position information of said second data corresponding toa feature of said focused data is read from said storage means, saidthird step regards said position information having the shortestdistance to a position of said focused data as position information ofsaid second data corresponding to said focused data.